Calibrating an image capture device with a detachable lens

ABSTRACT

In a method, it is determined that a detachable lens is mounted on an image capture device in a first orientation. A first image of a controlled scene is captured with the detachable lens mounted in the first orientation. It is determined that the detachable lens is mounted on the image capture device in a second orientation that is rotated approximately 180 degrees from the first orientation. A second image of the controlled scene is captured with the detachable lens in the second orientation. A first image circle center of the first image is determined. A second image circle center of the second image is determined. An average image circle center is determined, based on the first image circle center and the second image circle center. The average image circle center is provided to an image stabilization algorithm when the detachable lens is mounted on the image capture device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and the benefit of U.S. ProvisionalPatent Application No. 63/072,508, filed Aug. 31, 2020, the entiredisclosure of which is incorporated by reference herein.

TECHNICAL FIELD

This disclosure relates to a method for calibrating an image capturedevice with a detachable lens.

BACKGROUND

With some image capture devices, an additional (detachable) lens may bemounted on the device to provide additional features beyond those of abuilt-in (integrated) sensor-lens assembly (ISLA). For example, thedetachable lens may provide a wider field of view. When the detachablelens is mounted on the image capture device, the optical center of thefull system (image capture device plus mounted detachable lens) mayshift. This shift may result in image distortion or “wobble” introducedinto an image when image stabilization is performed.

SUMMARY

Disclosed herein are implementations of a method for calibrating animage capture device with a detachable lens.

One aspect of this disclosure is a method. It is determined that adetachable lens is mounted on an image capture device in a firstorientation. A first image of a controlled scene is captured with thedetachable lens mounted in the first orientation. It is determined thatthe detachable lens is mounted on the image capture device in a secondorientation that is rotated approximately 180 degrees from the firstorientation. A second image of the controlled scene is captured with thedetachable lens in the second orientation. A first image circle centerof the first image is determined. A second image circle center of thesecond image is determined. An average image circle center isdetermined, based on the first image circle center and the second imagecircle center. The average image circle center is provided to an imagestabilization algorithm when the detachable lens is mounted on the imagecapture device.

In aspects, the detachable lens produces an image circle on an imagesensor of the image capture device. In aspects, the detachable lens hasa symmetrical mechanical mounting on the image capture device. Inaspects, the detachable lens is a hemispherical lens. In aspects, thecontrolled scene is a scene with uniform lighting. In aspects, thecontrolled scene is a scene of a white background. In aspects, theaverage image circle center is usable by the image stabilizationalgorithm regardless of the orientation of the mounted detachable lens.In aspects, the method further includes storing the average image circlecenter in the image capture device.

Another aspect of this disclosure is a method. It is determined that adetachable lens is mounted on an image capture device in an orientation.An image of a controlled scene is captured with the detachable lens inthe orientation. The determining and the capturing are repeated for allpossible orientations of the detachable lens on the image capturedevice. An image circle center is determined for all of the capturedimages. An average image circle center is determined, based on all ofthe determined image circle centers. The average image circle center isprovided to an image stabilization algorithm when the detachable lens ismounted on the image capture device.

In aspects, the detachable lens produces an image circle on an imagesensor of the image capture device. In aspects, the detachable lens hasa symmetrical mechanical mounting on the image capture device. Inaspects, the detachable lens is a hemispherical lens. In aspects, thecontrolled scene is a scene with uniform lighting. In aspects, thecontrolled scene is a scene of a white background. In aspects, theaverage image circle center is usable by the image stabilizationalgorithm regardless of the orientation of the mounted detachable lens.In aspects, the method further includes storing the average image circlecenter in the image capture device.

Another aspect of this disclosure is a non-transitory computer-readablestorage medium including processor-executable routines that, whenexecuted by a processor, facilitate a performance of operations,including: receiving a first image of a controlled scene with adetachable lens mounted in a first orientation on an image capturedevice; receiving a second image of the controlled scene with thedetachable lens mounted in a second orientation on the image capturedevice, the second orientation different from the first orientation;determining a first image circle center of the first image; determininga second image circle center of the second image; determining an averageimage circle center based on the first image circle center and thesecond image circle center; and providing the average image circlecenter to an image stabilization algorithm when the detachable lens ismounted on the image capture device.

In aspects, the detachable lens produces an image circle on an imagesensor of the image capture device. In aspects, the detachable lens hasa symmetrical mechanical mounting on the image capture device. Inaspects, the average image circle center is usable by the imagestabilization algorithm regardless of the orientation of the mounteddetachable lens.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is best understood from the following detaileddescription when read in conjunction with the accompanying drawings. Itis emphasized that, according to common practice, the various featuresof the drawings are not to-scale. On the contrary, the dimensions of thevarious features are arbitrarily expanded or reduced for clarity.

FIGS. 1A-B are isometric views of an example of an image capture device.

FIG. 2 is a block diagram of electronic components of an image capturedevice.

FIGS. 3A-B are isometric views of an example image capture device with adetachable lens.

FIG. 4 is an isometric rear view of an example of a detachable lensusable with the image capture device shown in FIGS. 3A-B.

FIG. 5 is a diagram of an example of an image center difference betweendifferent mounting orientations of the detachable lens on the imagecapture device.

FIG. 6 is a flowchart of a method for calibrating an image capturedevice with a detachable lens.

DETAILED DESCRIPTION

With some image capture devices, an additional (detachable) lens may bemounted on the device to provide additional features beyond those of abuilt-in (integrated) sensor-lens assembly (ISLA). For example, thedetachable lens may provide a wider field of view or better imagestabilization. If the detachable lens is hemispherical, for example, itmay be possible to capture a full panorama image with the image capturedevice. In this instance, the image stabilization of the image capturedevice keeps the horizon level to assist in obtaining a smooth panoramicimage.

One example of a detachable lens is a hemispherical lens having a widefield of view, which may correspond to an image height of an imagesensor of the image capture device. The field of view of the detachablelens may allow for better image stabilization margins than with the ISLAof the image capture device for an identical output field of view. Withthe increased stabilization margin, the image stabilization algorithmmay perform full 360° horizon leveling.

The image stabilization algorithm, and in particular, the image warpingused in the image stabilization algorithm, is sensitive to a shift ofthe optical center. A shift in the optical center can create wobblyimages in a stabilized video, for example, since the stabilizationalgorithm may not be aware of the distortion change. There are threepossible sources of shift in the optical center: (1) the activealignment of the ISLA of the image capture device on the image sensor(e.g., the optical axis of the lens of the ISLA has to be aligned withthe center of the image sensor), (2) the ISLA mounting on the body ofthe image capture device and any mechanical defects of a mounting systemfor the detachable lens, and (3) static tilt of the detachable lens.Even if steps are taken to reduce the amount of optical center shiftfrom these three sources, it may still be possible to have an imagecapture device with an optical center shift of 50 pixels or more. Toknow the actual optical center shift of an image capture device, theoptical center needs to be calibrated. The calibrated optical center maythen be provided as an input to the image stabilization algorithm of theimage capture device to obtain better image stabilization.

To achieve image stabilization, the optical center of the full system(the image capture device plus the additional lens) needs to beobtained. Certain image stabilization algorithms are sensitive todisplacement of the optical center, and this displacement can causedistortion artifacts in the image (e.g., because the optical center hasshifted to the left or to the right). If image warping is performed, andif the optical center of the system does not match the optical centerused in the stabilization algorithm, the resulting image may include a“wobble” when the image capture device is moved by a user. To avoid the“wobble” in an image and to obtain the best possible imagestabilization, the optical center of the full system needs to beestimated. It is therefore desirable to have a method for calibratingthe image capture device with the additional lens attached.

FIGS. 1A-B are isometric views of an example of an image capture device100. The image capture device 100 may include a body 102, a lens 104structured on a front surface of the body 102, various indicators on thefront surface of the body 102 (such as light-emitting diodes (LEDs),displays, and the like), various input mechanisms (such as buttons,switches, and/or touch-screens), and electronics (such as imagingelectronics, power electronics, etc.) internal to the body 102 forcapturing images via the lens 104 and/or performing other functions. Thelens 104 is configured to receive light incident upon the lens 104 andto direct received light onto an image sensor internal to the body 102.The image capture device 100 may be configured to capture images andvideo and to store captured images and video for subsequent display orplayback.

The image capture device 100 may include an LED or another form ofindicator 106 to indicate a status of the image capture device 100 and aliquid-crystal display (LCD) or other form of a display 108 to showstatus information such as battery life, camera mode, elapsed time, andthe like. The image capture device 100 may also include a mode button110 and a shutter button 112 that are configured to allow a user of theimage capture device 100 to interact with the image capture device 100.For example, the mode button 110 and the shutter button 112 may be usedto turn the image capture device 100 on and off, scroll through modesand settings, and select modes and change settings. The image capturedevice 100 may include additional buttons or interfaces (not shown) tosupport and/or control additional functionality.

The image capture device 100 may include a door 114 coupled to the body102, for example, using a hinge mechanism 116. The door 114 may besecured to the body 102 using a latch mechanism 118 that releasablyengages the body 102 at a position generally opposite the hingemechanism 116. The door 114 may also include a seal 120 and a batteryinterface 122. When the door 114 is an open position, access is providedto an input-output (I/O) interface 124 for connecting to orcommunicating with external devices as described below and to a batteryreceptacle 126 for placement and replacement of a battery (not shown).The battery receptacle 126 includes operative connections (not shown)for power transfer between the battery and the image capture device 100.When the door 114 is in a closed position, the seal 120 engages a flange(not shown) or other interface to provide an environmental seal, and thebattery interface 122 engages the battery to secure the battery in thebattery receptacle 126. The door 114 can also have a removed position(not shown) where the entire door 114 is separated from the imagecapture device 100, that is, where both the hinge mechanism 116 and thelatch mechanism 118 are decoupled from the body 102 to allow the door114 to be removed from the image capture device 100.

The image capture device 100 may include a microphone 128 on a frontsurface and another microphone 130 on a side surface. The image capturedevice 100 may include other microphones on other surfaces (not shown).The microphones 128, 130 may be configured to receive and record audiosignals in conjunction with recording video or separate from recordingof video. The image capture device 100 may include a speaker 132 on abottom surface of the image capture device 100. The image capture device100 may include other speakers on other surfaces (not shown). Thespeaker 132 may be configured to play back recorded audio or emit soundsassociated with notifications.

A front surface of the image capture device 100 may include a drainagechannel 134. A bottom surface of the image capture device 100 mayinclude an interconnect mechanism 136 for connecting the image capturedevice 100 to a handle grip or other securing device. In the exampleshown in FIG. 1B, the interconnect mechanism 136 includes foldingprotrusions configured to move between a nested or collapsed position asshown and an extended or open position (not shown) that facilitatescoupling of the protrusions to mating protrusions of other devices suchas handle grips, mounts, clips, or like devices.

The image capture device 100 may include an interactive display 138 thatallows for interaction with the image capture device 100 whilesimultaneously displaying information on a surface of the image capturedevice 100.

The image capture device 100 of FIGS. 1A-B includes an exterior thatencompasses and protects internal electronics. In the present example,the exterior includes six surfaces (i.e. a front face, a left face, aright face, a back face, a top face, and a bottom face) that form arectangular cuboid. Furthermore, both the front and rear surfaces of theimage capture device 100 are rectangular. In other embodiments, theexterior may have a different shape. The image capture device 100 may bemade of a rigid material such as plastic, aluminum, steel, orfiberglass. The image capture device 100 may include features other thanthose described here. For example, the image capture device 100 mayinclude additional buttons or different interface features, such asinterchangeable lenses, cold shoes, and hot shoes that can addfunctional features to the image capture device 100.

The image capture device 100 may include various types of image sensors,such as charge-coupled device (CCD) sensors, active pixel sensors (APS),complementary metal-oxide-semiconductor (CMOS) sensors, N-typemetal-oxide-semiconductor (NMOS) sensors, and/or any other image sensoror combination of image sensors.

Although not illustrated, in various embodiments, the image capturedevice 100 may include other additional electrical components (e.g., animage processor, camera system-on-chip (SoC), etc.), which may beincluded on one or more circuit boards within the body 102 of the imagecapture device 100.

The image capture device 100 may interface with or communicate with anexternal device, such as an external user interface device (not shown),via a wired or wireless computing communication link (e.g., the I/Ointerface 124). Any number of computing communication links may be used.The computing communication link may be a direct computing communicationlink or an indirect computing communication link, such as a linkincluding another device or a network, such as the internet, may beused.

In some implementations, the computing communication link may be a Wi-Filink, an infrared link, a Bluetooth (BT) link, a cellular link, a ZigBeelink, a near field communications (NFC) link, such as an ISO/IEC 20643protocol link, an Advanced Network Technology interoperability (ANT+)link, and/or any other wireless communications link or combination oflinks.

In some implementations, the computing communication link may be an HDMIlink, a USB link, a digital video interface link, a display portinterface link, such as a Video Electronics Standards Association (VESA)digital display interface link, an Ethernet link, a Thunderbolt link,and/or other wired computing communication link.

The image capture device 100 may transmit images, such as panoramicimages, or portions thereof, to the external user interface device viathe computing communication link, and the external user interface devicemay store, process, display, or a combination thereof the panoramicimages.

The external user interface device may be a computing device, such as asmartphone, a tablet computer, a phablet, a smart watch, a portablecomputer, personal computing device, and/or another device orcombination of devices configured to receive user input, communicateinformation with the image capture device 100 via the computingcommunication link, or receive user input and communicate informationwith the image capture device 100 via the computing communication link.

The external user interface device may display, or otherwise present,content, such as images or video, acquired by the image capture device100. For example, a display of the external user interface device may bea viewport into the three-dimensional space represented by the panoramicimages or video captured or created by the image capture device 100.

The external user interface device may communicate information, such asmetadata, to the image capture device 100. For example, the externaluser interface device may send orientation information of the externaluser interface device with respect to a defined coordinate system to theimage capture device 100, such that the image capture device 100 maydetermine an orientation of the external user interface device relativeto the image capture device 100.

Based on the determined orientation, the image capture device 100 mayidentify a portion of the panoramic images or video captured by theimage capture device 100 for the image capture device 100 to send to theexternal user interface device for presentation as the viewport. In someimplementations, based on the determined orientation, the image capturedevice 100 may determine the location of the external user interfacedevice and/or the dimensions for viewing of a portion of the panoramicimages or video.

The external user interface device may implement or execute one or moreapplications to manage or control the image capture device 100. Forexample, the external user interface device may include an applicationfor controlling camera configuration, video acquisition, video display,or any other configurable or controllable aspect of the image capturedevice 100.

The user interface device, such as via an application, may generate andshare, such as via a cloud-based or social media service, one or moreimages, or short video clips, such as in response to user input. In someimplementations, the external user interface device, such as via anapplication, may remotely control the image capture device 100 such asin response to user input.

The external user interface device, such as via an application, maydisplay unprocessed or minimally processed images or video captured bythe image capture device 100 contemporaneously with capturing the imagesor video by the image capture device 100, such as for shot framing orlive preview, and which may be performed in response to user input. Insome implementations, the external user interface device, such as via anapplication, may mark one or more key moments contemporaneously withcapturing the images or video by the image capture device 100, such aswith a tag or highlight in response to a user input or user gesture.

The external user interface device, such as via an application, maydisplay or otherwise present marks or tags associated with images orvideo, such as in response to user input. For example, marks may bepresented in a camera roll application for location review and/orplayback of video highlights.

The external user interface device, such as via an application, maywirelessly control camera software, hardware, or both. For example, theexternal user interface device may include a web-based graphicalinterface accessible by a user for selecting a live or previouslyrecorded video stream from the image capture device 100 for display onthe external user interface device.

The external user interface device may receive information indicating auser setting, such as an image resolution setting (e.g., 3840 pixels by2160 pixels), a frame rate setting (e.g., 60 frames per second (fps)), alocation setting, and/or a context setting, which may indicate anactivity, such as mountain biking, in response to user input, and maycommunicate the settings, or related information, to the image capturedevice 100.

The image capture device 100 may be used to implement some or all of thetechniques described in this disclosure, such as the method 600described in FIG. 6 .

FIG. 2 is a block diagram of electronic components in an image capturedevice 200. The image capture device 200 may be a single-lens imagecapture device, a multi-lens image capture device, or variationsthereof, including an image capture device with multiple capabilitiessuch as use of interchangeable integrated sensor-lens assemblies. Thedescription of the image capture device 200 is also applicable to theimage capture devices 100 of FIGS. 1A-B.

The image capture device 200 includes a body 202 which includeselectronic components such as capture components 210, a processingapparatus 220, data interface components 230, movement sensors 240,power components 250, and/or user interface components 260.

The capture components 210 include one or more image sensors 212 forcapturing images and one or more microphones 214 for capturing audio.

The image sensor(s) 212 is configured to detect light of a certainspectrum (e.g., the visible spectrum or the infrared spectrum) andconvey information constituting an image as electrical signals (e.g.,analog or digital signals). The image sensor(s) 212 detects lightincident through a lens coupled or connected to the body 202. The imagesensor(s) 212 may be any suitable type of image sensor, such as acharge-coupled device (CCD) sensor, active pixel sensor (APS),complementary metal-oxide-semiconductor (CMOS) sensor, N-typemetal-oxide-semiconductor (NMOS) sensor, and/or any other image sensoror combination of image sensors. Image signals from the image sensor(s)212 may be passed to other electronic components of the image capturedevice 200 via a bus 280, such as to the processing apparatus 220. Insome implementations, the image sensor(s) 212 includes adigital-to-analog converter. A multi-lens variation of the image capturedevice 200 can include multiple image sensors 212.

The microphone(s) 214 is configured to detect sound, which may berecorded in conjunction with capturing images to form a video. Themicrophone(s) 214 may also detect sound in order to receive audiblecommands to control the image capture device 200.

The processing apparatus 220 may be configured to perform image signalprocessing (e.g., filtering, tone mapping, stitching, and/or encoding)to generate output images based on image data from the image sensor(s)212. The processing apparatus 220 may include one or more processorshaving single or multiple processing cores. In some implementations, theprocessing apparatus 220 may include an application specific integratedcircuit (ASIC). For example, the processing apparatus 220 may include acustom image signal processor. The processing apparatus 220 may exchangedata (e.g., image data) with other components of the image capturedevice 200, such as the image sensor(s) 212, via the bus 280.

The processing apparatus 220 may include memory, such as a random-accessmemory (RAM) device, flash memory, or another suitable type of storagedevice, such as a non-transitory computer-readable memory. The memory ofthe processing apparatus 220 may include executable instructions anddata that can be accessed by one or more processors of the processingapparatus 220. For example, the processing apparatus 220 may include oneor more dynamic random-access memory (DRAM) modules, such as double datarate synchronous dynamic random-access memory (DDR SDRAM). In someimplementations, the processing apparatus 220 may include a digitalsignal processor (DSP). More than one processing apparatus may also bepresent or associated with the image capture device 200.

The data interface components 230 enable communication between the imagecapture device 200 and other electronic devices, such as a remotecontrol, a smartphone, a tablet computer, a laptop computer, a desktopcomputer, or a storage device. For example, the data interfacecomponents 230 may be used to receive commands to operate the imagecapture device 200, transfer image data to other electronic devices,and/or transfer other signals or information to and from the imagecapture device 200. The data interface components 230 may be configuredfor wired and/or wireless communication. For example, the data interfacecomponents 230 may include an I/O interface 232 that provides wiredcommunication for the image capture device, which may be a USB interface(e.g., USB type-C), a high-definition multimedia interface (HDMI), or aFireWire interface. The data interface components 230 may include awireless data interface 234 that provides wireless communication for theimage capture device 200, such as a Bluetooth interface, a ZigBeeinterface, and/or a Wi-Fi interface. The data interface components 230may include a storage interface 236, such as a memory card slotconfigured to receive and operatively couple to a storage device (e.g.,a memory card) for data transfer with the image capture device 200(e.g., for storing captured images and/or recorded audio and video).

The movement sensors 240 may detect the position and movement of theimage capture device 200. The movement sensors 240 may include aposition sensor 242, an accelerometer 244, or a gyroscope 246. Theposition sensor 242, such as a global positioning system (GPS) sensor,is used to determine a position of the image capture device 200. Theaccelerometer 244, such as a three-axis accelerometer, measures linearmotion (e.g., linear acceleration) of the image capture device 200. Thegyroscope 246, such as a three-axis gyroscope, measures rotationalmotion (e.g., rate of rotation) of the image capture device 200. Othertypes of movement sensors 240 may also be present or associated with theimage capture device 200.

The power components 250 may receive, store, and/or provide power foroperating the image capture device 200. The power components 250 mayinclude a battery interface 252 and a battery 254. The battery interface252 operatively couples to the battery 254, for example, with conductivecontacts to transfer power from the battery 254 to the other electroniccomponents of the image capture device 200. The power components 250 mayalso include an external interface 256, and the power components 250may, via the external interface 256, receive power from an externalsource, such as a wall plug or external battery, for operating the imagecapture device 200 and/or charging the battery 254 of the image capturedevice 200. In some implementations, the external interface 256 may bethe I/O interface 232. In such an implementation, the I/O interface 232may enable the power components 250 to receive power from an externalsource over a wired data interface component (e.g., a USB type-C cable).

The user interface components 260 may allow the user to interact withthe image capture device 200, for example, providing outputs to the userand receiving inputs from the user. The user interface components 260may include visual output components 262 to visually communicateinformation and/or present captured images to the user. The visualoutput components 262 may include one or more lights 264 and/or moredisplays 266. The display(s) 266 may be configured as a touch screenthat receives inputs from the user. The user interface components 260may also include one or more speakers 268. The speaker(s) 268 canfunction as an audio output component that audibly communicatesinformation and/or presents recorded audio to the user. The userinterface components 260 may also include one or more physical inputinterfaces 270 that are physically manipulated by the user to provideinput to the image capture device 200. The physical input interfaces 270may, for example, be configured as buttons, toggles, or switches. Theuser interface components 260 may also be considered to include themicrophone(s) 214, as indicated in dotted line, and the microphone(s)214 may function to receive audio inputs from the user, such as voicecommands.

The image capture device 200 may be used to implement some or all of thetechniques described in this disclosure, such as the method 600described in FIG. 6 .

FIGS. 3A-B are isometric views of an example of an image capture device300 with a detachable lens 330. FIG. 3A shows the image capture device300 with the detachable lens 330 separated from a body 302 of the imagecapture device. FIG. 3B shows the image capture device 300 with thedetachable lens 330 attached to the body 302 of the image capture device300. Similar to the image capture device 100, the image capture device300 may include the body 302, an integrated sensor-lens assembly (ISLA)304 structured on a front surface of the body 302, various indicators onthe front surface of the body 302 (such as light-emitting diodes (LEDs),displays, and the like), various input mechanisms (such as buttons,switches, and/or touch-screens), and electronics (such as imagingelectronics, power electronics, etc.) internal to the body 302 forcapturing images via the ISLA 304 and/or performing other functions. TheISLA 304 is configured to receive light incident upon the ISLA 304 andto direct received light onto an image sensor internal to the body 302.The image capture device 300 may be configured to capture images andvideo and to store captured images and video for subsequent display orplayback.

The image capture device 300 may include a liquid-crystal display (LCD)or other form of a display 306 to show video, images, and/or statusinformation such as battery life, camera mode, elapsed time, and thelike. The image capture device 300 may also include a mode button 308and a shutter button 310 that are configured to allow a user of theimage capture device 300 to interact with the image capture device 300.For example, the mode button 308 and the shutter button 310 may be usedto turn the image capture device 300 on and off, scroll through modesand settings, and select modes and change settings. The image capturedevice 300 may include additional buttons or interfaces (not shown) tosupport and/or control additional functionality.

The image capture device 300 may include a microphone 312 on a frontsurface and may include other microphones on other surfaces (not shown).The microphone 312 may be configured to receive and record audio signalsin conjunction with recording video or separate from recording of video.

The image capture device 300 may also include additional features notshown in FIG. 3 , the additional features being similar to featuresrecited above in connection with FIG. 1 . For example, the image capturedevice 300 may also include an indicator to indicate a status of theimage capture device 300 and/or one or more speakers on a surface of theimage capture device 300 configured to play back recorded audio or emitsounds associated with notifications.

The image capture device 300 includes a mounting structure 320configured to receive the detachable lens 330. The mounting structure320 includes a base 322 attached to the body 302. A front collar 324extends perpendicularly from the base 322 and is generally annular inconfiguration. The front collar 324 includes a pair of radial mountingmembers 326 that are formed integrally with the front collar 324 and areconfigured for releasable engagement with the detachable lens 330 suchthat the detachable lens 330 is attachable to and detachable from theimage capture device 300 via the mounting structure 320, as described infurther detail below.

The detachable lens 330 includes an outer frame 332 surrounding a lensassembly 334. Additional detail on the construction of the detachablelens 330 is provided in connection with FIG. 4 .

FIG. 4 is an isometric rear view of an example of the detachable lens330 usable with the image capture device 300 shown in FIGS. 3A-B. Amounting plate 400 is connected to the outer frame 332 by a series offasteners 402. The lens assembly 334 is secured to the outer frame 332by an insert 404 and an inner frame 406. A sealing member 408 engageswith the front collar 324 of the image capture device 300 (shown in FIG.3 ) to create a watertight seal when the detachable lens 330 is attachedto the image capture device 300. The mounting plate 400 includes twoslots 410, 412 configured in correspondence with the radial mountingmembers 326 on the mounting structure 320 such that the radial mountingmembers 326 extend through the slots 410, 412 upon attaching thedetachable lens 330 to the image capture device 300.

The mounting plate 400 may also include an orientation indicator 414positioned in one of the slots 410, 412 (shown in FIG. 4 in slot 412).The orientation indicator 414 is used by a user of the image capturedevice 300 to determine the orientation of the detachable lens 330 whenattached to the image capture device 300. The orientation indicator 414may be any indicator that is visible to the user of the image capturedevice 300, and does not interfere with the operation of the detachablelens 330 or the image capture device 300.

FIG. 5 is a diagram of an example of an image center difference betweendifferent mounting orientations of a detachable lens on an image capturedevice, for example, the detachable lens 330 and the image capturedevice 300 shown in FIGS. 3A to 3B. An image sensor 500 of the imagecapture device 300 captures a first image 502 when the detachable lens330 is mounted on the image capture device 300. In FIG. 5 , the firstimage 502 is a represented as a circle with a dashed line. When thedetachable lens 330 is mounted in a first orientation, the first image502 has a first image circle center 504 represented as a cross with adashed line. When the detachable lens 330 is mounted in a secondorientation (e.g., rotated by approximately 180 degrees from the firstorientation), a second image 506 represented as a circle with a dottedline has a second image circle center 508 represented as a cross with adotted line. The distance between the two centers 504, 508 is referredto as an excursion 510 of the detachable lens 330. The excursion 510 isexpressed in pixels because the positions of the two centers 504, 508are estimated directly in the images 502, 506. The excursion 510 is onthe order of a few pixels, but is generally small enough that the imagestabilization algorithm of the image capture device 300 can absorb sucha difference. The centers 504, 508 are a representation in the images502, 506 of the optical axis of the full system (the detachable lens 330mounted on the image capture device 300).

FIG. 6 is a flowchart of a method 600 for calibrating an image capturedevice with a detachable lens, for example, the image capture device 300using the detachable lens 330 shown in FIGS. 3A to 3B. It is determinedthat the detachable lens 330 is mounted on the image capture device 300in a first orientation (operation 602). Based on the configuration ofthe mounting structure 320 of the image capture device 300 and thecorresponding slots 410, 412 of the detachable lens 330, it is possibleto orient the detachable lens 330 on the image capture device 300 in twodifferent positions rotated approximately 180 degrees from each other.To properly calibrate the image capture device 300, the detachable lens330 needs to be mounted in both orientations. After calibration, thedetachable lens 330 may be mounted on the image capture device 300 ineither orientation and the difference between the two orientations isaccounted for when the image capture device 300 performs imagestabilization.

After it has been determined that the detachable lens 330 is mounted onthe image capture device 300 in the first orientation (operation 602),the image capture device 300 captures a first image of a controlledscene (operation 604). As referred to herein, a “controlled scene” is asetting in which there is uniform lighting, for example, lighting via adiffuser or an image taken of a white background.

It is determined that detachable lens 330 is mounted on the imagecapture device 300 in a second orientation rotated approximately 180degrees from the first orientation (operation 606). After it has beendetermined that the detachable lens 330 is mounted on the image capturedevice 300 in the second orientation (operation 606), the image capturedevice 300 captures a second image of the controlled scene (operation608).

A first image circle center 504 of the first image 502 is determined(operation 610), and a second image circle center 508 of the secondimage 506 is determined (operation 612). It is noted that the firstimage may instead be the image 506 from FIG. 5 and that the second imagemay instead be the image 502 from FIG. 5 . It is also noted that anysuitable algorithm for determining an image circle center may be used inoperations 610 and 612. The choice of image circle center determinationalgorithm does not change the operation of the method 600.

An average image circle center is determined based on the first imagecircle center 504 and the second image circle center 506 (operation614), for example, by the following equation:

$\begin{matrix}{{ImageCircleCenter}_{Avg} = \frac{{ImageCircleCenter}_{1} + {ImageCircleCenter}_{2}}{2}} & {{Equation}\mspace{14mu}(1)}\end{matrix}$

where ImageCircleCenter_(Avg) is the average image circle center,ImageCircleCenter₁ is the first image circle center 504, andImageCircleCenter₂ is the second image circle center 506.

The average image circle center may be stored in the image capturedevice 300 in a nonvolatile memory. The average image circle center isused as an estimate of the optical center of the full system (imagecapture device 300 with the detachable lens 330 mounted) to account forcases where the ISLA 304 of the image capture device 300 is notperfectly centered within the mounting structure 320 of the imagecapture device 300, as misalignment may affect the position of theoptical center of the full system. Using the average image circle centeralso helps to avoid aggregating minor errors in alignment in any of theimaging components (e.g., the ISLA 304 of the image capture device 300,the mounting structure 320 of the image capture device 300, and thedetachable lens 330).

The average image circle center is provided to an image stabilizationalgorithm in the image capture device 300 when the detachable lens 330is mounted on the image capture device 300 (operation 616). It is notedthat in operation 616, because an average image circle center is used bythe image stabilization algorithm, the detachable lens 330 may bemounted on the image capture device 300 in either the first orientationor the second orientation, and the image stabilization algorithm willoperate correctly.

The image capture device 300 has a single point, a symmetry point, thatis independent from the detachable lens and that serves as an axis ofradial symmetry. This symmetry point is located close to the averageimage circle center. The symmetry point (e.g., the barycenter) for eachimage capture device is located close to the average image circlecenter. Each different image capture device is separately calibratedbecause each image capture device has its own symmetry point.

In some embodiments, the method 600 is performed on the production linefor each image capture device 300 made. In some embodiments, the method600 may be performed in the field by a user. This may become necessaryin circumstances where the image capture device 300 has been dropped(for example) and the detachable lens 330 is then attached to the imagecapture device 300. In such embodiments, the user identifies the problem(that the images obtained by the image capture device 300 exhibitdistortion artifacts) and performs the calibration. In some embodiments,the image capture device 300 may be configured to automatically detectwhen images start including distortion artifacts that can be correctedby performing the method 600, and the image capture device 300 thenalerts the user to perform the method 600. In some embodiments, it maybe possible to perform the method 600 with only one position image, butthe detachable lens 330 would need to not exhibit any differencesbetween the different mounting positions. In some embodiments, it may bepossible to eliminate operations 602 and 606 from the method 600 withoutaffecting the overall operation of the method 600.

In some embodiments, the mounting structure 320 of the image capturedevice 300 includes more than two radial mounting members 326. In thisconfiguration, the detachable lens 330 would include a correspondingnumber of slots (e.g., slots 410, 412) such that the radial mountingmembers fit into the slots to secure the detachable lens 330 onto themounting structure 320. In this configuration, the method 600 would bemodified to take an image of the controlled scene with the detachablelens 330 in all possible orientations of the detachable lens 330 on theimage capture device 300 (e.g., operations 602 and 604 are repeated forall possible orientations of the detachable lens 330 on the imagecapture device 300).

In some embodiments, a non-transitory computer-readable storage mediumincludes processor-executable routines that, when executed by aprocessor, facilitate a performance of the method 600.

This disclosure has used a hemispherical lens as an example of adetachable lens 330 in describing different embodiments. It is notedthat the method 600 may also be performed for different types ofdetachable lenses, as long as the detachable lens 330 has a symmetricalmechanical mounting and produces an image circle on the image sensor ofthe image capture device 300.

The method 600 is used to calibrate the symmetry point of the imagecapture device 300, and the calibrated symmetry point is inherent to theparticular image capture device 300. By storing the symmetry point (theaverage image circle center) in the image capture device 300, additionallenses (for example, a wide angle lens, a telephoto lens, or a macrolens) may be attached to the image capture device 300 without having torecalibrate the image capture device 300. In addition, being able tore-use the stored symmetry point benefits a user of the image capturedevice 300 because recalibrating the image capture device 300 forvarious lens types would typically include substantial additionalprocessing and may be a complex process that a user might find overlyburdensome.

While the method 600 has been described here as being able to providevideo stabilization improvements, the benefits of the method 600 mayalso address vignetting (radial shading). By centering color/luminanceshading using the calibrated symmetry center of the image capture device300, overall image quality may be improved.

While the disclosure has been described in connection with certainembodiments, it is to be understood that the disclosure is not to belimited to the disclosed embodiments but, on the contrary, is intendedto cover various modifications and equivalent arrangements includedwithin the scope of the appended claims, which scope is to be accordedthe broadest interpretation so as to encompass all such modificationsand equivalent structures as is permitted under the law.

What is claimed is:
 1. A method, comprising: determining that adetachable lens is mounted on an image capture device in a firstorientation; capturing a first image of a controlled scene with thedetachable lens mounted in the first orientation; determining that thedetachable lens is mounted on the image capture device in a secondorientation that is rotated approximately 180 degrees from the firstorientation; capturing a second image of the controlled scene with thedetachable lens in the second orientation; determining a first imagecircle center of the first image; determining a second image circlecenter of the second image; determining an average image circle centerbased on the first image circle center and the second image circlecenter; and providing the average image circle center to an imagestabilization algorithm when the detachable lens is mounted on the imagecapture device.
 2. The method of claim 1, wherein the detachable lensproduces an image circle on an image sensor of the image capture device.3. The method of claim 1, wherein the detachable lens has a symmetricalmechanical mounting on the image capture device.
 4. The method of claim1, wherein the detachable lens is a hemispherical lens.
 5. The method ofclaim 1, wherein the controlled scene is a scene with uniform lighting.6. The method of claim 1, wherein the controlled scene is a scene of awhite background.
 7. The method of claim 1, wherein the average imagecircle center is usable by the image stabilization algorithm regardlessof the orientation of the mounted detachable lens.
 8. The method ofclaim 1, further comprising: storing the average image circle center inthe image capture device.
 9. A method, comprising: determining that adetachable lens is mounted on an image capture device in an orientation;capturing an image of a controlled scene with the detachable lens in theorientation; repeating the determining and the capturing for allpossible rotatable orientations of the detachable lens on the imagecapture device; determining an image circle center for all of thecaptured images; determining an average image circle center based on allof the determined image circle centers; and providing the average imagecircle center to an image stabilization algorithm when the detachablelens is mounted on the image capture device.
 10. The method of claim 9,wherein the detachable lens produces an image circle on an image sensorof the image capture device.
 11. The method of claim 9, wherein thedetachable lens has a symmetrical mechanical mounting on the imagecapture device.
 12. The method of claim 9, wherein the detachable lensis a hemispherical lens.
 13. The method of claim 9, wherein thecontrolled scene is a scene with uniform lighting.
 14. The method ofclaim 9, wherein the controlled scene is a scene of a white background.15. The method of claim 9, wherein the average image circle center isusable by the image stabilization algorithm regardless of theorientation of the mounted detachable lens.
 16. The method of claim 9,further comprising: storing the average image circle center in the imagecapture device.
 17. A non-transitory computer-readable storage mediumcomprising processor-executable routines that, when executed by aprocessor, facilitate a performance of operations comprising: receivinga first image of a controlled scene with a detachable lens mounted in afirst orientation on an image capture device; receiving a second imageof the controlled scene with the detachable lens mounted in a secondorientation on the image capture device, the second orientationdifferent from the first orientation; determining a first image circlecenter of the first image; determining a second image circle center ofthe second image; determining an average image circle center based onthe first image circle center and the second image circle center; andproviding the average image circle center to an image stabilizationalgorithm when the detachable lens is mounted on the image capturedevice.
 18. The non-transitory computer-readable storage medium of claim17, wherein the detachable lens produces an image circle on an imagesensor of the image capture device.
 19. The non-transitorycomputer-readable storage medium of claim 17, wherein the detachablelens has a symmetrical mechanical mounting on the image capture device.20. The non-transitory computer-readable storage medium of claim 17,wherein the average image circle center is usable by the imagestabilization algorithm regardless of the orientation of the mounteddetachable lens.